Move muckrake

Initial patch for KIP-25

Note that to install ducktape, do *not* use pip to install ducktape. Instead:

```
$ git clone gitgithub.com:confluentinc/ducktape.git
$ cd ducktape
$ python setup.py install
```

Author: Geoff Anderson <geoff@confluent.io>
Author: Geoff <granders@gmail.com>
Author: Liquan Pei <liquanpei@gmail.com>

Reviewers: Ewen, Gwen, Jun, Guozhang

Closes #70 from granders/KAFKA-2276 and squashes the following commits:

a62fb6c [Geoff Anderson] fixed checkstyle errors
a70f0f8 [Geoff Anderson] Merged in upstream trunk.
8b62019 [Geoff Anderson] Merged in upstream trunk.
47b7b64 [Geoff Anderson] Created separate tools jar so that the clients package does not pull in dependencies on the Jackson JSON tools or argparse4j.
a9e6a14 [Geoff Anderson] Merged in upstream changes
d18db7b [Geoff Anderson] fixed :rat errors (needed to add licenses)
321fdf8 [Geoff Anderson] Ignore tests/ and vagrant/ directories when running rat build task
795fc75 [Geoff Anderson] Merged in changes from upstream trunk.
1d93f06 [Geoff Anderson] Updated provisioning to use java 7 in light of KAFKA-2316
2ea4e29 [Geoff Anderson] Tweaked README, changed default log collection behavior on VerifiableProducer
0eb6fdc [Geoff Anderson] Merged in system-tests
69dd7be [Geoff Anderson] Merged in trunk
4034dd6 [Geoff Anderson] Merged in upstream trunk
ede6450 [Geoff] Merge pull request #4 from confluentinc/move_muckrake
7751545 [Geoff Anderson] Corrected license headers
e6d532f [Geoff Anderson] java 7 -> java 6
8c61e2d [Geoff Anderson] Reverted jdk back to 6
f14c507 [Geoff Anderson] Removed mode = "test" from Vagrantfile and Vagrantfile.local examples. Updated testing README to clarify aws setup.
98b7253 [Geoff Anderson] Updated consumer tests to pre-populate kafka logs
e6a41f1 [Geoff Anderson] removed stray println
b15b24f [Geoff Anderson] leftover KafkaBenchmark in super call
0f75187 [Geoff Anderson] Rmoved stray allow_fail. kafka_benchmark_test -> benchmark_test
f469f84 [Geoff Anderson] Tweaked readme, added example Vagrantfile.local
3d73857 [Geoff Anderson] Merged downstream changes
42dcdb1 [Geoff Anderson] Tweaked behavior of stop_node, clean_node to generally fail fast
7f7c3e0 [Geoff Anderson] Updated setup.py for kafkatest
c60125c [Geoff Anderson] TestEndToEndLatency -> EndToEndLatency
4f476fe [Geoff Anderson] Moved aws scripts to vagrant directory
5af88fc [Geoff Anderson] Updated README to include aws quickstart
e5edf03 [Geoff Anderson] Updated example aws Vagrantfile.local
96533c3 [Geoff] Update aws-access-keys-commands
25a413d [Geoff] Update aws-example-Vagrantfile.local
884b20e [Geoff Anderson] Moved a bunch of files to kafkatest directory
fc7c81c [Geoff Anderson] added setup.py
632be12 [Geoff] Merge pull request #3 from confluentinc/verbose-client
51a94fd [Geoff Anderson] Use argparse4j instead of joptsimple. ThroughputThrottler now has more intuitive behavior when targetThroughput is 0.
a80a428 [Geoff Anderson] Added shell program for VerifiableProducer.
d586fb0 [Geoff Anderson] Updated comments to reflect that throttler is not message-specific
6842ed1 [Geoff Anderson] left out a file from last commit
1228eef [Geoff Anderson] Renamed throttler
9100417 [Geoff Anderson] Updated command-line options for VerifiableProducer. Extracted throughput logic to make it reusable.
0a5de8e [Geoff Anderson] Fixed checkstyle errors. Changed name to VerifiableProducer. Added synchronization for thread safety on println statements.
475423b [Geoff Anderson] Convert class to string before adding to json object.
bc009f2 [Geoff Anderson] Got rid of VerboseProducer in core (moved to clients)
c0526fe [Geoff Anderson] Updates per review comments.
8b4b1f2 [Geoff Anderson] Minor updates to VerboseProducer
2777712 [Geoff Anderson] Added some metadata to producer output.
da94b8c [Geoff Anderson] Added number of messages option.
07cd1c6 [Geoff Anderson] Added simple producer which prints status of produced messages to stdout.
a278988 [Geoff Anderson] fixed typos
f1914c3 [Liquan Pei] Merge pull request #2 from confluentinc/system_tests
81e4156 [Liquan Pei] Bootstrap Kafka system tests

added close method to kstreamjob

Adding secure and securityConfigFile args to javaapi SimpleConsumer

implement StateStore methods in SlidingWindow

BUG-40771. Kafka metrics are not being shown on the dashboard.

KAFKA-2979: Add phase3 to rolling upgrade adding ACLs [SASL protocols…

Author: Michael Blume <mike@loggly.com>

Reviewers: Guozhang Wang

Closes #4 from MichaelBlume/patch-1

(cherry picked from commit af0df09)
Signed-off-by: Guozhang Wang <wangguoz@gmail.com>

kafka fixes

Prevent scenario where no replicas in ISR are leader eligible

…che#4)

Travis couldn't access one of the initially encrypted variables for some reason.

… Travis (apache#4)

TICKET =
LI_DESCRIPTION =

Travis couldn't access one of the initially encrypted variables for some reason.

EXIT_CRITERIA = MANUAL ["describe exit criteria"]

Checkstyle fixes

TICKET =
LI_DESCRIPTION =

[LI-HOTFIX] Add build changes to publish github LinkedIn Kafka artifacts to bintray (#1)
TICKET =
LI_DESCRIPTION =

[NOTE] This is a temporary measure to publish artifacts until CI is properly set up to do the job automatically. Users are not expected to run this themselves.

EXIT_CRITERIA = MANUAL ["describe exit criteria"]

[LI-HOTFIX] Add changes for CI builds and publishing artifacts to bintray. (#2)
TICKET =
LI_DESCRIPTION =

Travis will kick off a build and publish artifacts to bintray upon creating a tag in the "x.y.z.w" format.

EXIT_CRITERIA = MANUAL ["describe exit criteria"]

[LI-HOTFIX] Try different encrypted bintray-related env variables for Travis (apache#4)
TICKET =
LI_DESCRIPTION =

Travis couldn't access one of the initially encrypted variables for some reason.

EXIT_CRITERIA = MANUAL ["describe exit criteria"]

[LI-HOTFIX] Set Bintray-related env variables via repository setting instead of in .travis.yml (apache#5)
TICKET =
LI_DESCRIPTION =

EXIT_CRITERIA = MANUAL ["describe exit criteria"]

[LI-HOTFIX] Use the maven repo under the LinkedIn Bintray account to publish artifacts (apache#25)
TICKET =
LI_DESCRIPTION =

EXIT_CRITERIA = MANUAL ["describe exit criteria"]

EXIT_CRITERIA = MANUAL ["describe exit criteria"]
# EXIT_CRITERIA = <EXIT_CRITERIA> <PARAMETERS>
# e.g.,
# when the specified hash(s) is presented in the history, this commit is no longer needed:
# EXIT_CRITERIA = HASH [<hash1>, <hash2> ...]
# When the specified tickets are closed and there are patches with these tickets in the title in the commit history, this commit is no longer needed:
# EXIT_CRITERIA = TICKET [<ticket1>, <ticket2> ...]
# The exit criteria for this commit requires manual investigation:
# EXIT_CRITERIA = MANUAL [<description>]

merge latest changes from Apache/kafka to microsoft/kafka trunk

…ache#14)

More abstraction of the Fetcher code to split into classes to buffer raw results (FetchBuffer) and to collect raw results into ConsumerRecords (FetchCollector).

…ache#14)

More abstraction of the Fetcher code to split into classes to buffer raw results (FetchBuffer) and to collect raw results into ConsumerRecords (FetchCollector).

fetch when source cluster leader change